A Structured Wrapper Induction System for Extracting Information from Semi-Structured Documents

We propose an extensible architecture which allows wrapper-learning systems to be easily constructed and tuned. In this architecture the bias of the wrapper-learning system is encoded as an ordered set of “builders”, each associated with some restricted extraction language L. To implement a new builder it is only necessary to implement a small set of core operations for L. Builders can also be constructed by combining other builders. A single master learning algorithm which invokes the builders handles most of the real work of learning. The learning system described here is fully implemented, and is part of an “industrial-strength” wrapper-learning system which has been used to extract job postings from more than 500 sites.